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PRELIMINARY AMENDMENT 
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IMPROVEMENTS IN AND RELATING TO SURVEILLANCE SYSTEMS 

This invention relates to the field of surveillance, and more specifically, 
but not exclusively to surveillance based systems which utilize intelligence of 
knowledge based systems to support their operation. 

Improvements in the art of closed circuit image acquisition cameras have 
provided for a marked increase in the use of remote cameras for applications 
such as shop and public area security and more evidently traffic monitoring and 
law enforcement. 

Typically, the type of camera systems utilised for shop and public area 
security fall into two main categories, namely the fixed field of view, fixed focus 
real time recorded video system, and the more labour intensive human 
operated system, usually comprising an operator directable and focusable 
camera assembly. 

Law enforcement applications for camera systems typically comprise 
either a similar type operator based system as described above for traffic 
monitoring, or fixed position speed cameras, which utilize a radar based object 
speed detection sensor and timing unit to capture images against fixed 
references to allow for subsequent human analysis for the purpose of court 
evidence. Increasingly the use of digital camera technology is providing for 
more sophisticated vehicle speed analysis cameras, but typically such systems 
are mobile and require a human operator. 

In an attempt to provide the police with greater information about the 
movement of vehicles around the UK road system, tracking systems have been 
proposed which utilize camera systems that effectively capture and analyse 
images of passing car registration plates. The information captured is relayed to 
a central unit, the information then being searchable so as to allow the selective 
identification of a known vehicle's movements. 

One common problem encountered in the art is that of the amount of 
data which is required to be stored to enable meaningful post analysis of the 
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captured scene. Data compression techniques have been applied such as the 
storing of only every fourth video image frame, but this leads to the step 
animated appearance of some surveillance system image playback. 

A further problem associated with surveillance systems is the quality of 
images and information discernible from such images that are captured and 
stored. The requirement to have a field of view camera or other imaging means 
sufficient to cover a whole scene usually dictates the use of low resolution, 
wide-angle equipment. Evidence of the poor quality images usually associated 
with surveillance systems can be illustrated by the footage produced from 
security systems such as those used in high street banks and petrol stations. 
Often the facial features of criminals are difficult to identify using the results of 
such systems, even when the criminal is captured facing directly at a camera. 

Our invention provides for surveillance and monitoring system which 
comprises the ability to combine state of the image capture technology and real 
time scene and image analysis, with an intelligent, knowledge based system. 
This results in a surveillance system which requires minimum operator 
intervention to provide an intelligent assessment of captured real time events, 
and further provides for the ability to prompt an operator or initiate certain 
control functions on the witnessing of some pre-determined event or series of 
events. 

Accordingly there is a surveillance system comprising imaging means, 
said imaging means being positioned so as to have a field of view of an area 
over which surveillance is required, image processing means, said image 
processing means being programmable such that an operator can pre- 
determine the events which activate use of image analysis means, image 
analysis means, said image analysis means utilising processing means to 
determine suitable control function responses to events viewed by said imaging 
means, and system control means, said system control means providing 
general control functions to said imaging means, scene processing means and 
scene analysis means. 
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In a preferred embodiment of said invention, said imaging means may 
comprise at least one first imaging means and at least one second imaging 
means, said second imaging means having the ability to provide more detailed 
information in relation to a partial section of the view of said first imaging 
means. 

In another preferred embodiment of said invention, a plurality of said 
surveillance systems are so positioned in relation to an area which is to be 
subject to surveillance, such that the combination of the fields of view of said 
imaging systems provides for the ability to follow the progress of event moving 
or developing within the field of view of the plurality of said imaging means, 
thereby providing for a target tracking ability, allowing event analysis to be 
conducted over an increased time. 

In a further embodiment of the invention said first and second imaging 
means are movable either in response to commands from a central system 
control means, or in response to operator issued commands. 

In yet a further embodiment of the invention, said image analysis means 
comprises an artificial intelligence (Al) means, said Al means providing for 
increased selectivity in the choosing of target objects and patterns of events, 
thereby further reducing the level of data storage or compression required for 
operation of said surveillance system. 

Embodiments of the invention will now be described by way of example 
only with reference to the following figures ; 

Figure 1 is a schematic representation of a surveillance system in 
accordance with the invention. 

Figure 2 is a diagrammatic representation of a scene captured by said 
first and said second cameras. 

Figure 3 is a diagrammatic representation of a network of cameras in 
accordance with the invention. 

Figure 1 shows a surveillance system 1 in accordance with the invention 
in which an imaging means 3 comprises two charged couple device (CCD) 
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cameras 2 & 4. Camera 2 comprises a wide to normal field of view low to 
normal resolution CCD camera, whereas in comparison to camera 2, camera 4 
comprises a narrow field of view, higher resolution camera. Camera 4 may 
further comprise a zoom facility and may or may not be aligned to the boresight 
of camera 2. Cameras 2 and 4 may also be provided with dependent or 
independent controllable movement and rotation about a range of axis to further 
facilitate the possibility of acquiring an image for analysis. 

Additionally cameras 2 and 4 may comprise, but are not limited to, 
discrete camera systems, multiple lenses or multiple mirror systems or adaptive 
optics systems, which may be co-located, widely spaced or be part of a multiple 
camera application. 

The outputs of cameras 2 and 4 are fed to the image processing means 
8 via a system control means 6. The image processing means 6 provides the 
first level of image analysis by processing the images captured by said camera 
2 and determining whether further analysis of the image is required by 
re f erence to at least one database means 12. The image processing means 
will have been pre-programmed to be triggered into requesting further analysis 
of an image or images on witnessing certain pre-determined events, features or 
sequences of actions or images. The pre-determination of these events etc 
may require the use of and access to a range of data-base means, possibly via 
a hub means 10, including, but not limited to, human operators, Rule Based 
Systems (RBS), Knowledge Based Systems (KBS), Artificial Intelligence 
Systems (Al), Data-Bases, Algorithms and the like. 

Examples of images or events which may form the basis of a pre- 
programmed trigger to request further analysis include the speed, dimensions, 
temperature, luminescence, time, frequency of passing, conformity or 
abnormality of human beings, animals, vehicles or other defined objects. 

Additionally the image processing means may make a qualitative 
assessment of the image provided by camera 2, and should the threshold 
quality of the image not be provided by camera 2, then the image processing 
means will request from the system control means 6 that the second camera 4 
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be used so as to provide higher quality images for passing to the image 
analysis means 14. The assessment of the adequacy of the image to ensure 
sufficient characteristics have been captured will ensure the increase in 
certainty of recognition required for evidence purposes in criminal prosecutions. 

Additionally the image analysis could be utilised to monitor patterns of 
behaviour of individuals or groups of individuals for the purposes of marketing 
and/or sales or customer/product intelligence. 

The image analysis means 14 provides the surveillance system 1 with 
the ability to make a detailed analysis of the content of images supplied by 
cameras 2 and 4, such analysis being provided by access to a range of data- 
base means 18, in a similar fashion to that described above in connection with 
said image processing means. The image analysis means 14 is shown having 
a hub connection 16 to a range of data-base sources 18, which may include, 
but are not limited to human operators, RBS, KBS, Al, Data-bases and 
Algorithms and the like. The various database means available to the image 
analysis means may or may not include those database means accessible by 
the image processing means. 

The image analysis means will have the ability to demand further image 
information, such information comprising higher or lower resolution images from 
cameras 2 and 4 or wider angle images along with use of additional imaging 
means such as, but not limited to, conventional radar, synthetic aperture radar 
(SAR), infra-red imaging systems (visible and non-visible), milimetric wave 
imaging systems, acoustic and magnetic systems. 

Operator defined criteria will define the images which will be selected for 
image analysis 14. The nature and extent of the image analysis conducted will 
depend on the functionality of the particular surveillance system and it's 
constitute components. Examples of the extent of image analysis include the 
facial identification of human beings, with possible cross references to 
databases of known criminals, suspects or other persons whose whereabouts is 
required to be established, car registration analysis, with cross references to 
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details such as vehicle colour, speed, location, occupants, frequency of passing 
a known location, or time spent within a particular catchment area. 

The image analysis function of the surveillance system is designed to 
include an element of 'self-learning' or Al which will provide for the 
establishment of pattern or behavioural analysis. Such analysis will provide for 
an effective filter mechanism to eliminate from further analysis activities or 
patterns of behaviour which it has been previously established are not worthy of 
further analysis, for example, the same car seen every Wednesday at 5:00pm, 
to 5:30pm, the same two men enter the building society carrying bags every 
Friday, a person of the same dimensions passes wearing a crash helmet every 
day. 

The use of the image analysis means 14 combined with the other 
elements of the system will enable the identification of discrete features or 
known patterns of behaviour which differ from the established norm, or may 
warrant surveillance on the basis of matching with the identification of a 
particular person, object or combination thereof. 

The system control means 6 provides the surveillance system 1 with 
control functions required to administer the operation of the system as a whole, 
and may include, but are not limited to self diagnostic functions (image quality, 
system functionality, communications integrity etc), alarm functions relating to 
unlawful interference with the system and control of the imaging means by 
functions such as managing requests from the image processing means to 
switch or alter characteristics or functions of the imaging means (i.e. direction, 
focus, lens, resolution, type of camera etc). Additionally the system control 
means may control and/or manage the communications links to the various 
external bodies, databases, establishments or persons required by the type of 
operation to which the system is being put to use. 

Figure 2 shows an example of a scene as may be viewed by an imaging 
system 2 & 4 as described above. The extent of the viewable image is shown 
24, the view comprising a section of highway 28 and an area of public walkway 
40. A car 26 is shown passing through the scene 24, and in this example of a 
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surveillance system in accordance with the invention, a trigger has been pre- 
programmed within the image processing means 8 such that when a car passes 
through the scene 24, the image analysis means 14 is activated and certain 
information is sought for further analysis. 

In the car 26 passing through the scene 24, the image analysis means 
would typically be programmed so as to try and acquire positive identification of 
the numerals displayed on the vehicle registration plate 32. Should the image 
supplied by the camera 2 not provide sufficient clarity of contain sufficient 
information for such data to be acquired, then the image processor means will 
request, via the system control means, that a higher resolution camera means 4 
is used, giving more detailed information relating to one area of the scene 24. 
In this example, frame 30 shows a representative fields of view for camera 4, 
the information contained therein being sufficient for the image analysis means 
14 to make a positive identification of the vehicle registration plate. Once the 
image has been acquired, the image analysis means 14 can connect via the 
hub 16 to a database of known vehicle registrations, and if required can store or 
supply information relating to the said vehicle to another database or to any 
other type of display of data storage means. Additionally, other databases may 
be interrogated so as to provide further identification features such as the cross 
matching of the vehicle colour to that on the registration details, cross 
referencing the registration to vehicles reported stolen or the owners of which 
are required for questioning by the police or insurance companies, along with 
other possible checks such as vehicle speed, frequency of passing a certain 
location and elapsed time spent within a certain vicinity. 

The system can further utilised for example to acquire images of vehicle 
occupants. Frame 34 depicts the field of view of camera 4, giving a view of the 
occupant of the vehicle. The image can then be interrogated by reference to a 
facial identification database (i.e.18) , which again may be further interfaced to 
additional databases and/or RBS, KBS and Al systems to enable the 
identification and knowledge of the movements and type of transport used by 
specific persons or groups of persons. 
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Figure 2 also depicts a view taken from an imaging means 2 showing a 
group of people 44 walking along a pathway 40. One of the pre-determined 
triggers for the image processing means 8 could be the recognition of 'human 
shaped objects' 44. The image analysis means would then be tasked with 
identifying the face of the human being by applying the use of camera 4 and 
frame 46. In the event that the face of the human could not be identified due to 
obscuration by means such as a hat 42 or balaclava (not shown), the image 
analysis means would record a failure to identify a target and depending on the 
nature and extent of the rules governing the database means may either raise 
an alarm to an operator, or begin a further sequence of analysis to try and 
further identify characteristics of the subject. Such further analysis could extend 
to identifying objects 50 carried by the subject, and by applying the use of 
higher resolution camera means 4, using field of view frame 48, the image 
analysis means will attempt to identify the object 50 by reference to a list of 
known threats, i.e. firearms, truncheons, knives etc. Depending on the result of 
the analysis, images may be stored and/or operator alarms maybe activated or 
other control functions initiated such as alarms, door closures, lighting 
increased and the like. 

Figure 3 shows a network of surveillance systems 52,54,56,58, in 
accordance with the invention. Each of said individual surveillance systems 
52,54,56,58 can operate in accordance with the system described in figure 1 , 
each having imaging means 2, 4, system control means 6, image processing 
means 8, image analysis means 14, hub means 10, 16 and database means 
12, 18. Additionally there is provided a master system control means 62, 
master image processing means 64, master image analysis means 66, and 
corresponding hub means 68, 70 and database means 72, 74. The master 
elements of the system 62, 64, 66 as depicted in figure 3, enable the 
surveillance system as a whole to be centrally managed or controlled to enable 
more sophisticated surveillance tasks to be undertaken. A typical connectivity is 
illustrated between the elements of the networked system, but various 
combinations or interface means may be provided between said elements such 
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as databus technology, digital or analogue cable or terrestrial transmissions or 
satellite communications. 

Utilising the invention in the embodiment shown at figure 3, if image 
analysis means 6 comprising a camera acquires an image which is passed by 
the image processing means 8 to the image analysis means 14, then should 
said image move out of view of the system 52, then via the master system 
control means 62, the imaging means of system 54 can be employed to 
continue the surveillance and analysis as required. This 'hand-over" between 
the individual elements of the networked system provides for a far greater area 
coverage than the individual system, and additionally provides for ensuring the 
continuity of surveillance required for evidence in criminal prosecutions. 

Alternatively, in an embodiment not illustrated, the surveillance system 
may be provided with a system control means and/or a master system control 
means which is provided with a plurality of imaging means connected to directly 
to said master or system control means, thereby providing for the centralising of 
the control functions of the system, and the probable reduction in the number of 
image processing and image analysis means required to perform the 
surveillance operation. 

The imaging means 3 provided for use with the system may be mobile, 
or connected or installed on a mobile platform or vehicle. 

The system as a whole may itself be self contained such that references 
to databases should be interpreted as references to algorithms contained in or 
accessible by the system. Such self contained systems may make use of 
mobile or fixed cameras but be self contained in the sense of no 
communications to databases outside of the system. 
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CLAIMS 



A surveillance system comprising imaging means , said imaging means 
being positioned so as to have a field of view of an area over which 
surveillance is required, image processing means, said image processing 
means being programmable such that an operator can pre-determine the 
events which activate use of image analysis means, image analysis 
means, said image analysis means utilising processing means to 
determine suitable control function responses to events viewed by said 
imaging means, and system control means, said system control means 
providing general control functions to said imaging means, scene 
processing means and scene analysis means. 

A surveillance system as claimed in claim 1 , wherein said imaging means 
comprising at least one first imaging means and at least one second 
imaging means, said second imaging means having the ability to provide 
more detailed information in relation to a partial section of the view of said 
first imaging means. 

A surveillance system as claimed in claims 1 and 2 above, wherein a 
plurality of said surveillance systems are so positioned in relation to an 
area which is to be subject to surveillance, that the combination of the 
fields of view of said imaging systems provides for the ability to follow the 
progress of event moving or developing within the field of view of the 
plurality of said imaging means. 

A surveillance system as claimed in claims 1 , 2 and 3 above, wherein said 
first and second imaging means are movable either in response to 
commands from a central system control means, or in response to 
operator issued commands. 

An imaging means for surveillance as claimed in claims 1 , 2 and 3 above, 
wherein said image analysis means comprises an artificial intelligence (Al) 
means. 

A surveillance system as hereinbefore described with reference to the 
accompanying drawings. 
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ABSTRACT 



IMPROVEMENTS IN AND RELATING TO SURVEILLANCE SYSTEMS 

A surveillance system which allows an operator to determine the type of 
objects or events which trigger image capture and analysis. Analysis of 
captured images is provided by access to various type of databases thereby 
providing for positive identification and monitoring of objects or events, and 
further provided for the system to issue control commands or prompt 
operations if required. 
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Q Hosmer 1 30184^Robert W. Faris,J£1352l_Richard G. Besha , 22770; M ark E. Nusbaum^3234ajvlichael J. Keenan, 32106; B ryan H. Davidson, 
O 30251; Stanley C. Spooner^ZSSS; Leonard C. Mitchard . 29009; Duane M. Byers, 33363; Paul J. Henon, 33626; Jeffry H Nelson , 30481; John R. 
""Tastova ^149; H . Warren Burnam, Jr^2^g6LThomas E. Bvrne . 32205: Mary J. Wilson. 32955 ; J. Scott Davidson, 33489j_ Alan M. Kagen, 36178; 
William J. Griffin, 31260; Robert A. Mola m29834; B. J. Sadoff, 36663; James D. Berquist, 347767'U pdeep S. Gill, 37334. 



X 



Inventor's Signature: 
Inventor: 

Residence: (city) 
Post Office Address: 
(Zip Code) 

Inventor's Signature: 
Inventor: 

Residence: (city) 
Post Office Address: 
(Zip Code) 



1olap ft29834; B. J. Sado 



Ml 




1 



Date: 



TILSLEY 
(last) 



British 
(citizenship) 



Stevenage (state/country) Hertfordshire, United Kingdom 

MATRA BAE Dynamics (UK) Limited, Six Hills Way, Stevenage, Hertfordshire. SG1 2 DA, United Kingdom fj7 K /\ 
SG1 2DA 

Date: 



(first) 



Ml 



(last) 



(citizenship) 



(state/country) 



FOR ADDITIONAL INVENTORS, check box □ and attach sheet with same information and signature and date for each. 
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